Conventionally, a background screen is used in taking an image in a photo studio, etc. However, in such image taking, a large background screen is used to generate a sufficient background, and a large image taking space is required. Furthermore, it also requires a laborious operation in exchanging a background screen, etc., and results in a costly image-taking process.
In this situation, it is desired to develop a small image-taking system which requires no large background screen or image taking space as in the conventional techniques so that a user can more easily enjoy taking images. In such a small image-taking system, since an image is taken in front of a provisional background in place of the background screen, it is proposed to perform a background replacing process of replacing a background in a taken image obtained by an image-taking process with a background desired by a client.
As an example of a technique of the background replacing process, for example, an image of a subject is taken with a background panel, etc. painted blue, etc. as a background of the subject, a portion of the same color as the background panel and a portion of a different color are discriminated from each other respectively as the background and the subject in the image obtained in the image-taking process, and the portion discriminated as the background in the taken image is replaced with a desired background. This process is known as a chroma key process.
In this chroma key process, when the shadow of the subject is taken on the background panel in the taken image, the shadow can prevent the subject from being discriminated from the background. Accordingly, a technique of taking an image is proposed (for example, refer to the patent document 1) using a background panel which emits light from a surface thereof as a background when an image as a target of the chroma key process is to be taken with the shadow of the subject on the background panel suppressed by the light emitted by the background panel. Furthermore, the patent document 1 also proposes a process of combining a portion discriminated as a subject with a desired background in the chroma key process by adjusting the color tone of the portion discriminated as the subject into the color tone depending on the color tone of the desired background, thereby generating a composite image in which the subject appears naturally with the desired background.
There is also a technique of taking an image of a subject in an open place as the background of the subject such as an outdoor background, etc. under two illumination conditions of the state in which the illumination is turned on toward the subject and the state in which the illumination is turned off, acquiring two taken images, discriminating in the two taken images the portions indicating different brightness levels from the portions indicating similar brightness levels as the subjects and the backgrounds of the taken images, respectively, and replacing the portion discriminated as the background in the taken image with a desired background (for example, refer to the patent document 2).    [Patent Document 1]    Japanese Patent Laid-Open No. 2000-224410 (pages 3 to 10, FIG. 3)    [Patent Document 2]    Japanese Patent Laid-Open No. 10-210340 (pages 2 to 3, FIG. 1)
To replace the background in a taken image with a desired background, it is necessary to appropriately discriminate between a subject in the taken image obtained in an image-taking process and its background. However, when a subject in a taken image includes a portion of the same color as a background panel in the technique described in the patent document 1 or used in the general chroma key process, the portion is erroneously recognized as a background, and there is the possibility that an unnatural background-replaced image can be generated by replacing a part of the subject replaced with the desired background.
In the technique described in the Patent Document 2, since a subject is discriminated from its background based on the brightness in the taken image, there is no possibility that a portion of the same color as its background can be erroneously recognized as a background. However, in a small desired image-taking system in which a user can enjoy taking a picture in a photo studio, etc., it is inevitable that image taking equipment such as a background panel, etc. is placed at the back of a subject. If the technique of the patent document 2 is applied to such a desired small image-taking system, the illuminating light emitted to the subject is reflected by the image taking equipment. As a result, in the two taken images obtained by the technique of the patent document 2, there is the possibility that other portions than a subject can have different brightness values, causing part of the background to be erroneously recognized as part of the subject, thereby generating an unnatural background-replaced image in which the part of the background is not replaced.
For example, in a taken image including the image of a person, there is the probability that the background appears through the hair of the person around the boundary between the head portion of the person and the background. When the clothes of the person is made of, for example, translucent fabric, there can be the possibility that the background appears through the translucent fabric around the boundary between the person and the background. Thus, there are cases in which a boundary between a subject and its background in a taken image includes an intermediate portion that cannot be clearly discriminated as the subject or the background. For example, when an intermediate portion is automatically discriminated as a background and replaced with a desired background, an unnatural background-replaced image in which a part of the subject is missing can be generated. If such an intermediate portion is automatically discriminated as a subject, an unnatural background-replaced image in which the intermediate portion contains the old background can be generated.
The present invention has been developed to solve the problems above, and it is an object of the present invention to provide an image-taking system capable of taking an image of a subject, acquiring a taken image, and generating a background-replaced image in which the background in the taken image is naturally replaced with a desired background.